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EVALUATION 1 


Evaluation keeps an organization on course and prevents it from ''flying blind". 
Evaluation is simply the process of finding honest and objective answers for questions 
like: How well are we doing what we aredoing? Evaluation can be used to judge 

or measure attitudes and ideas, programs, projects and even goals. These judge- 
ments have to be based on objective evidence and they have to be made systematically. 


Steps in Evaluation 


To take a simple case, suppose a community group is planning to raise enough funds 
to purchase an encyclopedia for the school library. The evaluation process would 
work in this way. 


1. What is the target? The committee with the job of fund raising will probably first 
survey the need and decide which encyclopedia to buy. If the group is convinced, on 
the basis of facts, workers available and other factors involved, it can raise the 
$500 needed to buy the fine encyclopedia the experts have recommended, they have 
set the goal. 


2. Where are we now? Once they have the goal they have a standard against which 
to measure their progress. It will be important to know what results their methods 
are producing. A careful record of progress toward the objective will also be a 
useful guide to future planning. 


3. What activities will lead toward the goal? The committee must now decide what 
form their fund raising will take. They decide on a Christmas bazaar because they 
feel it will attract many people in the community. The committee begins the search 
for volunteers to provide the items for sale. After a week, the committee members 
report little success. A careful analysis of information reveals that the Christmas 
bazaar will be poorly attended because other community affairs are planned for the 
same week. On the basis of the evidence the committee now changes its course of 
action. It decides to plan for a community party well after Christmas — a Valentine 
party in fact. The available facilities are surveyed, a program plan drawn up, 
publicity arranged and tickets distributed for sale. 


4, How are we doing? Here is where records of the effectiveness of various methods 
used in the past will be useful. Suppose, after ten days the sale is disappointing 
compared with the committee's goal and realistic expectations. The members will 
have to look for possible reasons. 


5. What does the collected evidence mean? A study of the record of the ticket sale, 
comparisons with past sales and other evidence may show that few people are talking 
about the project or that some large group, such as teenagers are not much interested. 


6. Are we doing what we should be doing? An increased number of canvassers, a 
telephone "blitz and a rash of posters are arranged to give more adequate publicity. 
The program of the party itself might be changed to include more features to attract 
teenagers or other groups who have not been buying tickets. 


7, Can we reach the objective? If it becomes clear that none of the committee's 
efforts can possibly produce the needed $500, the goal for the project may have to 
be cut down, perhaps to half. The committee would then begin to plan another 
project to be carried out later. The objective for the new project would be the 
other $250 needed to buy the encyclopedia. The committee might decide to modify 
its objective in other ways. For instance cut it by one-third and buy a cheaper 


encyclopedia. 


Another Application of the Steps 


For another example, suppose that at a Home and School Association meeting in 
November, interest is expressed in setting up an adult education program. Evaluation 
will play an important role in the project. 


Before members decide their goals they will probably try to determine the specific 
adult education needs in the community. They must consider the local facilities 
and whether leadership is available. They must decide how many classes will be 
held and at what times. 


Eventually, they might offer three courses at classes held from 7:30 to 9:30 on 
Weanesday evenings, beginning the second Wednesday in January and ending the 
third Wednesday in April (15 weeks) with a minimum enrolment of 15 people in 

each class. 


Having determined their goal, the committee can measure their progress. Allowing 
for the Christmas rush they have about six weeks to get the project under way. They 
might appoint a committee to meet once a week to follow up registrations, arrange 
for staff and class-rooms and take care of routine details. 


Now, the committee may notice early in December that there is little enthusiasm for 
one of the courses. Perhaps only four or five persistent talkers had originally 
wanted it. Meanwhile, the committee has seen that another course will require 
three or four leaders to take care of the numbers registering for it. 


The committee must now find solutions to two problems. It can make a closer 
canvass for participants in the unpopular course, substitute a more appealing subject, 
or limit the project to the two popular courses. The association may go outside the 
community to find enough leaders for the over-subscribed course or they may adopt 

a study-group approach that requires no lecturers. 


The committee may face decisions of this nature up to the week before the courses 
begin. As new evidence turns up, the committee will interpret it and modify decisions 
and actions in the light of new facts. Some association members may feel they are 
settling for less than the ideal goals set in advance. However, only frustration will 
result from continuing to strive for objectives that are unobtainable on the basis of 
good, sound evidence. 


Once the classes start, a new evaluation cycle begins. Each instructor may apply 
our seven steps in evaluation to find out how well the classes are working toward 

the goal. The evidence they analyze and interpret is made up of the opinions ex- 
pressed in various ways by members of the classes. When the classes are com- 
pleted in April, the committee will do an evaluation of the entire course, the methods 
of presentation, the times chosen, the facilities and suggestions for next year. In 
short, good evaluation is a continuous process. 


Principles of Evaluation 


Good evaluation meets people where they are. The ways in which a group of people 
think and the interest they take in any subject will vary with their average level of 
education and typical background experience. Standards used in evaluation should 
be those that will put the next improving step within easy reach of the group. It 
would be unwise to apply the standards of the London or Broadway stage to the 
productions of an amateur dramatic group at a local church. 


The best evaluation is co-operative. Sharing the work of measuring progress leads 
all members toward deeper understanding and greater desire to improve than if 
some higher authority carries out the evaluation for them. 


Useful evaluation considers every angle. Evaluators must consider everything 
related to the activities or undertaking that is being evaluated. The key factor may 
turn out to be one of the least obvious and something no one had considered. 


Business-like evaluation is continuous. No community organization or committee 
should wait until failure looms before they apply systematic evaluation. Different 
techniques may be used at different stages, but evaluation should be always going on. 


Effective evaluation uses frank and probing questions. Evaluation must criticize 

and diagnose not just activities, but also the ultimate goals to which the activities 
are directed. Goals we set for ourselves are not sacred. They should be reviewed 
periodically in the light of new experience and information. The purpose of evalua- 
tion is more than just finding out whether a desired result is or is not being achieved. 
Its aim is also to find out why a given course of action does or does not produce the 
desired result. 


All evaluation leads to action. The only purpose of evaluation is to improve programs 
and activities aimed at goals and to improve the goals themselves in the light of 
individual and social needs. 


Who Evaluates ? 


Because participation in community organizations is voluntary, it is good to think of 
evaluation as a group activity. The members should evaluate their own actions, no 
outsider should decide whether they have succeeded or failed. Outside experts are 

useful mainly in helping to devise methods for collecting the required kind of infor- 
mation and in helping the group to analyze and interpret that information. The final 
judgment or evaluation should be made by the group. 


What is Evaluated ? 


In a community organization, evaluation is often concerned with the organization 
itself, or the effects of the organization's work on its members and on the 
community. 


The organization needs to know how well it functions. What proportion of members 
attend meetings regularly? How well do members participate in the organization's 
activities? Is the organization making full and good use of all its resources of 
leadership and finances? How effective are its committees ? 


Evaluation of the organization's activities in terms of their effect on members will 
help keep the programs in tune with members'needs. Are there changes in the 
knowledge of members or in their behavior as members, neighbors and citizens ? 

Are they growing more mature and objective about their own and others behavior ? 
The harmony, or lack of it, among the members, and the effectiveness of the methods 
being used to deal with tensions are other factors that may be tested. 


The impact of the organization on the community may also be measured. How far is 
the organization helping to meet community needs? Does it have an appreciable 
effect on the co-operation among groups ? 


Evaluation and Goals 


Before they can determine the effectiveness of meetings, programs or activities, 
the members must have clear targets or goals. If these goals are too general, eval- 
uation will be difficult. For instance, a group that plans a series of meetings "to 
promote good citizenship" will find it hard to measure any progress. But a series 
of meetings planned "'to introduce new Canadians to community" has a well-defined 
target. 


In setting goals there are several questions to bear in mind. 


Are the goals in line with the aims, objects and policies of 
the organization? 


Are they acceptable to the members concerned? Are they expressed in 
words that members really understand ? 


Are they clear enough to be practical ? 


Is the idea contained in the goal in line with the way most of the members 
think? Does it set too high or too low a standard for this particular group ? 


Are the goals attainable? Can they be measured? 


Evaluation Techniques 


There are no hard and fast rules in evaluation. The measurement of progress is 
simple enough if all we have to do is collect statistics — for example, 


the number of people who buy tickets for a dance or enrol in 

a workshop, 

the number of people who respond to a fund-raising campaign, 

and 

the totals of donations by days or weeks, 

the number of votes for and against questions submitted to members. 


But more often than not, community groups want to find out things that cannot be 
measured so easily — for example, 


whether the members are satisfied with particular programs, policies 
or administrative procedures , 


what the members think about a community or group problem, 


whether the members are ready for a proposed change. 


Our most valuable source of evidence in evaluating community group activities is 
the freely expressed opinions of the members. The techniques for getting this 
evidence can vary from the most informal remarks to the most elaborate surveys. 
What is most important is that the problem should be approached from every angle 
so that evidence from one source may be compared with the evidence from another. 
This approach increases the objectivity and soundness of the conclusion. 


There are many devices for getting to know the views and feelings of a group. 

One of the simplest is the buzz-group, a technique for involving the members in 
program planning. At each meeting, the members may form groups of six or eight 
without leaving their places. Each buzz-group chooses a spokesman and after about 
ten minutes discussion he will report the program suggestions of his group to the 
general meeting. It is the spokesman's responsibility to ensure that everyone in the 
small group has a chance to contribute. 


Many organizations use post-meeting reaction slips. At the end of the meeting or 
program to be evaluated, a copy is handed to each member so that he can indicate, 
without signing his name, his reactions to specific questions. Great care must be 
taken in framing the questions so that they will not be answered thoughtlessly or 
automatically. 


The answers should also form a useful basis for planning future activities. Often 
some form of rating scale is needed so that the members can register shades of 
Opinion in between "yes"! and 'no" or say how much they liked or disliked a program, 
plan or idea. Excellent-good-fair-poor is a rating scale of four parts. A scale to 
rate the reaction of a group may also be divided into five parts, or three or ten, for 
that matter. A reaction scale in five parts might be attached to a question such as 
"How satisfactory did you find this lecture?" 


Completely Unsatisfactory 


satisfactory 


Evaluation in Program Planning 


Good programming is perhaps the most important single factor in the vitality of any 
community organization. Let's consider an example. 


Suppose the chairman of a program committee is a former public health nurse with 
an enthusiastic interest in preventive health care. She arranges a series of meetings 
— all lectures on the topic in which she is interested. She invites a well-qualified 
speaker to give the lectures. She and her committee make suitable physical arrange- 
ments for the meetings. The committee gets out special invitations and encloses 
pamphlets giving background information. 


The speaker is a highly trained person who speaks at great length. During the first 
talk the members are restless. At the second meeting there is a drop in attendance. 
For the third meeting, only a handful appear. The series is an obvious failure. 


Applying some of the principles of evaluation, the failure may be analyzed: 


Evaluation of the plan would have shown that the method of presentation 
was out of line with the members way of thinking. 


Through co-operative evaluation at the planning stage, the members 
would have had the opportunity to express their opinions. 


Continuous evaluation would have provided the warning signals, and 
suggested ways of saving the later meetings from failure. 


Functional evaluation, at any stage, would have led to a change of the 
program content or presentation. 


Before deciding on their program plan, the committee might have asked the members 
whether they would like a series of lectures ona single topic. If that had been 
acceptable, a list of topics could have been drawn up (with spaces for suggestions) 
and each member given an opportunity to rate his choices on the list. The topics 
selected would then have been in keeping with the members expressed wishes. 


After the first lecture, a general post-meeting questionnaire like the one shown 
might have been offered to the members. 


ASSESSMENT OF PROGRAM 


1. Please indicate your opinion of this program 


(a) poor excellent 


(b) Comments 


Committee convenors and executive officers should apply the principles of 
evaluation to their own work. Their starting point is the question, ''What 
should we be doing?'' When an executive group has a clear, honest answer to 
that question, it may begin to collect evidence to show where they are in relation 
to the goal. 


The effectiveness of an executive can be measured in rated answers to key 
questions: 


What proportion of the organization's programs are 
initiated by the members rather than the officers ? 


How many members maintain their interest at a high level 


and participate actively in the programs ? 


They may use some of the methods already suggested to find out the members' 
opinions of their work. 


The Use of an Observer 


Committees and executive officers might also ask an experienced outside 
observer to report after watching them in action for some time. A person who 
is not involved in the processes of the organization can often see them more 
clearly than the participants. 


Questions the observer would ask are: 


Does the group understand its purpose? 


Is the group progressing toward its objectives? If not, why? 


Is the group making the best possible use of all the resources 
and facilities available to it? They may "belong" to the community 
or others, but can be shared. 


Does the leadership of the organization help all members to 
participate and to take increasing responsibility ? 


The observer's report should be presented frankly and objectively. It is un- 
necessary to refer to people by name or to criticize individuals. The observer's 
responsibility is to describe the group process as he saw it. The group itself can 
draw the conclusions about the weakness or strength of its own procedures. 


